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HUMAN  FACTORS  AND  SAFETY  ASSESSMENT  OF  ARMY  SYSTEMS 


INTRODUCTION 


Ceneral 


This  is  a  summary  final  report  that  documents  the  events  over  the  term  of 
Contract  No.  MDASC3-86-C-034 1 .  The  report  does  not  attempt  to  provide  a 
detailed  account  of  contractor  performance  of  each  task,  but  only  to  summarize 
them.  More  detailed  information  regarding  the  AN/TRC-170  test  is  contained  in 
FH  WP  8803,  regarding  the  Multiple  Subscriber  Equipment  in  FH  WP  8801,  and 
regarding  the  M1A1  FOE  In  a  Research  Note  that  has  not  completed  the  publication 
steps.  The  SINCGARS  publication  awaits  the  completion  of  the  test,  the  OTEA 
report,  and  the  ARI  report. 


Background 


At  certain  milestones  during  the  acquisition  cycle  of  U.S.  Army  materiel, 
test  end  evaluation  (T&E)  is  performed  to  ensure  that  the  design  specifications 
are  met  and  the  materiel  provides  the  required  operational  utility.  This  T&E 
is  usually  comprised  of  two  types,  developmental  test  and  evaluation  (DT&E)  and 
operational  test  and  evaluation  (OT&E).  DT&E  is  the  responsibility  of  the  Army 
Material  Command  (AMC),  specifically  the  U.S.  Army  Test  and  Evaluation  Command 
(USATECOM) ,  and  OT&E  is  the  responsibility  of  the  U.S.  Army  Operational  Test 
and  Evaluation  Agency  (USAOTEA)  and  the  various  test  boards  such  as  the 
Communications  Electronics  Board  and  the  Artillery  Board. 

USAOTEA's  mission  is  to  provide  an  independent  assessment  of  the  operational 
utility  of  systems  defined  as  major  acquisitions.  Data  for  this  assessment  is 
collected  by  exercising  the  system  using  Army  personnel  who  are  representative 
of  the  expected  user  population  in  operationally  realistic  but  controlled 
scenarios.  The  intent  is  to  determine  the  adequacy  of  the  system  to  support 
operations  and  complete  its  mission  In  a  tactical  environment.  Typical  test 
issues  include 

1.  Mission  performance, 

2.  Adequacy  of  the  Table  of  Organization  and  Equipment, 

3.  Vulnerability, 

4.  Reliability,  Availability,  and  Maintainability, 

5.  Logistics  supportability , 

6.  Adequacy  of  training  and  technical  manuals, 

7.  Human  factors, 

8.  Safety. 

In  recent  years,  the  last  three  issues  have  become  much  more  critical  and 
visible,  due  primarily  to  the  increasing  complexity  of  the  Army  tactical 
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systems.  New  systems  being  developed  and  fielded  are  incorporating  state-of- 
the-art  technology  into  their  design,  especially  computer  technology.  This 
complexity  has  so  increased  the  task  demands  placed  on  the  operators  and 
maintainers  that  operational  effectiveness  is  being  jeopardized.  This,  along 
with  the  limitations  on  quality  and  quantity  of  personnel  in  the  manpower 
inventory,  has  created  significant  concerns  within  the  Army.  The  appearance 
and  implementation  of  the  MANPRINT  initiative  and  of  the  continuous  and 
comprehensive  evaluation  (C2E)  concept  are  responses  to  these  concerns. 

Army  Regulation  602-2,  "Manpower  and  Personnel  Integration  (MANPRINT)  in 
Materiel  Acquisition  Process"  was  effective  as  of  18  May  1987.  It  establishes  a 
requirement  for  a  MANPRINT  Management  Plan.  MANPRINT  is  an  umbrella  concept 
integrating  manpower,  personnel,  training,  human  factors  engineering,  health 
hazards,  and  system  safety.  The  intent  of  MANPRINT  is  to  support  and  influence 
systems  design  and  associated  support  requirements  so  that  developmental,  non- 
developmental ,  and  product-improved  systems  can  be  operated  and  maintained  cost 
effectively  and  safely,  and  consistently  with  manpower  structures,  personnel 
aptitudes,  skills,  and  training  resource  constraints.  Thus,  appropriate 
implementation  of  MANPRINT  would  ensure  that  factors  such  as  limitations  in  the 
quality  and  quantity  of  personnel  available  in  the  current  and  future  Army 
personnel  pool,  and  training  requirements  are  considered  and  explicitly  planned 
for  in  the  system  design  and  acquisition.  The  development  of  MANPRINT  methodol¬ 
ogy,  subsuming  human  factors  and  safety  (as  in  this  contract),  is  a  current  ARI 
research  task. 

AR  602-2  directs  the  CG  of  OTEA  to: 

•  Collect  and  evaluate  soldier  performance  data  (time  and  accuracy)  for 
all  critical  operations  and  maintenance  tasks  in  major  and  designated 
acquisition  program  testing  and  evaluation  for  which  OTEA  is  respon¬ 
sible.  (See  AR  70-10  and  AR  71-3.) 

•  Include  soldier  performance  data  on  critical  operations  and  maintenance 
tasks  in  any  calculations  of  system  effectiveness  and  availability 
presented  to  ASARC  reviews. 

•  Assess  effectiveness  of  proposed  training  for  critical  operations  and 
maintenance  tasks. 

•  Verify  that  soldiers  used  in  testing  are  representative  of  the  user 
population  as  defined  in  the  target  audience  description." 

This  initiative  ties  in  directly  with  the  concept  of  C2E.  OTEA  is  the 
Army's  lead  organization  for  the  development  and  dissemination  of  C2E.  It 
involve^,  the  collection  and  collation  of  data  from  a  variety  of  sources  such  as 
USATECOM,  contractor,  USAOTEA,  and  the  Human  Engineering  Laboratory  into  a 
single,  integrated  data  base  that  will  be  used  to  evaluate  systems  from  the 
conceptual  design  phase  through  and  beyond  deployment.  The  intent  of  this 
concept  is  to  develop  a  historical  data  base  that  can  be  used  to  evaluate 
systems  and  provide  lessons  learned  information  for  those  having  responsibili¬ 
ties  for  the  acquisition  of  future  systems. 
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USAOTEA  and  ARI  (e.g.,  the  Fort  Hood  Field  Unit)  had  an  agreement  dating 
from  1983  wherein  the  Field  Unit  provided  in-house  support  for  the  collection 
of  human  factors  and  safety  data  during  selected  (limited)  OTEA  operational 
•■ests.  This  support  was  increased  by  the  transfer  of  OTEA  funds  to  ARI  and 
ARI's  procuring  contractor  support  to  the  effort.  The  present  contract  employs 
CMA  funds,  that  being  required  when  testing  is  at  the  Follow-on  stage.  During 
the  period  of  this  contract,  the  MANPRINT  and  C2E  emphases  have  become  urgent, 
and  there  has  been  pressure  to  change  (enhance)  the  test  support  to  meet  the 
methodological  requirements  of  MANPRINT  and  C2E,  hardly  feasible  with  the 
existing  contract  funding.  OTEA  fund  citing  has  ceased;  OTEA  will  do  its  own 
contracting  for  support  of  future  tests. 

Objectives 


The  objectives  of  this  contract  were  to  provide  USAOTEA  with  human  factors 
applied  to  follow-on  test  methodology  development,  field  data  collection,  data 
analysis,  and  report  preparation.  Initially,  there  were  six  follow-on  evalua¬ 
tions  that  were  to  be  supported;  AN/AN/TRC-170  Troposcatter  Radio  Follow-On  Test 
and  Evaluation  (FOTE),  SINCGARS  V  Follow-On  Test  and  Evaluation  (FOTE),  Mobile 
Subscriber  Equipment  (MSE)  FOTE,  Maneuver  Control  System  (MCS)  FOE,  M1A1  Main 
Battle  Tank  FOE,  and  Single  Subscriber  Terminal  (SST)  FOE.  Of  these  six  tasks, 
the  SST  FOE  was  cancelled  and  the  MCS  FOE  was  shifted  beyond  the  scope  of  the 
contract.  This  report  summarizes  the  support  provided  for  the  other  four  tasks. 
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TEST  SUPPORT  SUMMARIES 


AN/TRC-170  Troposcatter  Radio  FOTE 


Background 

There  is  a  DoD  requirement  for  a  high-speed,  secure,  digital  transmission 
system  that  can  accommodate  the  volume  of  digital  voice  and  data  traffic  that 
is  anticipated  in  the  tactical  battlefield  environment  of  the  1980s  and  1990s. 
The  AN/TRC-170  Troposcatter  Radio  has  been  developed  to  meet  a  portion  of  the 
requirement . 

The  AN/TRC-170  V2  and  V3  sets  will  be  major  elements  in  the  Army's  com-  mica- 
tions  system  called  TRI-TAC.  They  are  designed  to  fill  the  requirement  for 
long-haul  trunking  of  communications  between  major  communications  nodes.  Set  V2 
provides  a  maximum  effective  range  of  150  miles  using  a  9  ft.  diameter  antenna. 
Set  V3  provides  a  maximum  effect  range  of  100  miles  using  a  6  ft.  diameter, 
trailer  mounted  antenna  system.  Both  sets  provide  128  to  2048  Kilobits  per 
second  transmission  rate  capability  in  the  4.4  to  5.0  GHz  frequency  band.  The 
V2  set  is  housed  in  one  S-280  shelter  and  one  low  profile  pallet.  The  V3  set  is 
housed  in  one  S-250  shelter  and  one  M-116A  trailer  chassis. 

During  the  months  of  September  1986  and  January  1987,  a  Follow-On  Test  and 
Evaluation  (FOTE)  of  the  AN/TRC-170  was  performed  at  Ft.  Huachuca,  AZ.  To  this 
point  in  system  development,  the  US  Air  Force,  as  lead  service  in  the  procure¬ 
ment,  had  been  responsible  for  developmental  and  operational  testing.  The  FOTE 
represented  the  US  Army's  first  total  and  independent  assessment  of  the  AN/TRC- 
1 70  in  the  Army  tactical  operating  environment,  with  soldiers. 

Objectives 


The  objectives  of  the  MANPRINT  assessment  of  the  AN/TRC-170  were  to  identify 
any  aspects  of  the  system  design  that  required  refinements  in  terms  of  human 
factors,  safety,  health  hazard,  training,  and  manpower  considerations. 

Personnel  was  not  a  TRAD0C  issue  for  this  test. 

Method 


The  player  personnel  for  the  test  consisted  of  35  operators  and  three 
maintainers.  MANPRINT  data  was  collected  using  the  following  four  types  of 
data  collection  methods. 

•  Structured  interviews  were  administered  to  the  operators,  maintainers, 
test  directorate,  unit  commanders,  and  data  collectors. 

•  Noise  levels  over  the  500  to  2000  hertz  range  were  measured  in  the 
shelters  during  high  power  operations,  using  an  sound  level  meter  and 
octave  band  analyzer. 


4 


•  Observational  data  was  collected  by  experienced  MANPRINT  personnel. 

•  Performance  times  were  measured  for  crew  performance  of  emplacement  and 
displacement  tasks.  These  tasks  were  the  most  critical  and  frequently 
performed.  Forty-two  separate  measures  were  collected.  Performance  was 
measured  using  the  standard  four  man  crews,  and  with  three  man  crews. 
Performance  data  regarding  three  man  crews  was  collected  to  see  if  there 
could  be  any  manpower  savings. 

In  addition,  a  MANPRINT  scoring  conference  was  held  to  assess  the  impact  on 
performance  for  each  MANPRINT  finding,  and  to  determine  the  priority  for 
correction  of  each  finding.  This  is  a  new  term  for  what  had  been  called  the 
Data  Analysis  Group's  meeting(s).  DAGs  were  divided  into  those  personnel 
concerned  with  the  validity  of  the  data  collected  pertaining  to  RAM,  and  those 
concerned  with  the  validity  of  the  data  pertaining  to  Human  Factors.  This 
conference  consisted  of  representatives  of  USAQTEA,  the  AN/TRC-170  Program 
Manager's  Office,  TRADOC,  and  the  manufacturer. 

Resul ts 


In  general,  the  system  performed  well  as  a  radio  relay.  Twenty-four  system 
deficiencies  were  identified,  with  the  most  critical  listed  below.  Detailed 
findings  are  contained  in  FH  WP  8803,  "MANPRINT  Test  Report  of  the  Follow-on 
Test  and  Evaluation  of  the  AN/TRC-170  (V)",  by  Sam  Bowser  and  Larry  Lyons,  4  Feb 
88. 


HF  Radio.  There  is  no  capability  for  high  frequency  radio  communication 
between  the  AN/TkC-170  and  other  tactical,  logistic  and  maintenance  units. 

M720  Shelter  Mobillzer.  The  shelter  mobilizer  (M720)  lacks  high  enough 
ground  clearance  for  effective  cross-country  travel  and  the  load  capacity  of 
its  cargo,  leading  to  equipment  damage.  Also,  the  mobilizer  cannot  be  moved  in 
reverse  due  to  the  height  of  the  hitch  and  the  angle  of  the  mobilizer  tongue. 

Generators .  The  two  10-kw  generators  provided  with  the  TOE  cannot  provide 
continuous  power  during  switchover,  and  cannot  automatically  adjust  for  load 
variations. 

Noise .  The  noise  levels  were  measured  at  89  dB(,a)  in  the  6belter«  during 
high  power  operations  with  the  doors  closed.  This  exceeds  the  85  dB(a)  criteria 
from  MIL-STD-1474 ,  and  requires  that  hearing  protection  be  worn  at  all  times  and 
limits  voice  communication  to  occasional  shouted  communications  within  two  feet, 
unsatisfactory  for  a  communications  system. 

Tailgates .  The  truck  tailgates  do  not  have  steps  and  handrails  for  safe 
ingress  to  and  egress  from  the  shelter. 

Low  profile  pallet.  The  lifting  of  the  low-profile  pallet,  containing  the 
9.5  foot  antenna,  requires  more  than  three  men.  If  the  crew  is  composed  of 
only  three,  there  is  a  high  chance  of  Injury. 
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Plan jar .  When  the  crew  were  using  the  Pionjar,  which  is  a  handheld  drill 
similar  to  a  jack  hammer  used  for  drilling  holes  and  inserting  stakes  in  the 
ground,  the  noise  levels  exceeded  110  dB(a).  The  ear  plugs  supplied  for 
hearing  protection  were  not  adequate. 

Training.  No  training  was  provided  as  part  of  the  26D6  MOS  training  for  the 
an/trc-i7o  on  the  use  of  the  Pionjar,  which  could  lead  to  inappropriate  use  and 
injury. 

Emplacement  and  displacement.  The  TRADOC  criteria  for  system  emplacement 
was  240  minutes  for  the  V 2  and  120  minutes  for  the  V3.  This  was  from  the  time 
the  unit  drove  onto  the  site  until  they  were  in  communication.  The  criterion 
for  displacement  was  120  minutes  for  the  V2.  There  was  no  displacement 
criterion  for  the  V3.  As  Table  1  illustrates,  the  crew  mean  performance  times 
exceeded  these  criteria  in  all  cases.  This  was  primarily  due  to  several  very 
difficult  tasks,  including  the  antenna  alignment. 


Table  1 


Mean  Performance  Times  (minutes)  for  Emplacement  and  Displacement  of  the 
AN/TRC-170,  V2  and  V3 


Emplacement  Displacement 


Equipment 

Crew  Size 

Criterion 

Mean 

Criterion 

Mean 

V2 

3 

240 

494 

120 

147 

4 

360 

134 

V3 

3 

120 

223 

None 

N/A 

4 

194 

A  tuber  of  difficulties  were  experienced  in  the  collection  of  the  perfor¬ 
mance  data.  First,  the  data  collection  was  all  manual.  Therefore,  the 
performance  times  could  only  be  sampled  due  to  limited  coverage  by  MANPRINT 
personnel.  Second,  data  was  only  collected  during  Phase  3  of  a  three  phase 
test  (Phase  1  was  the  pilot  and  Phase  2  was  record  testing)  due  to  resource 
constraints.  The  changes  in  crew  behavior  and  learning  curves  could  not  be 
determined.  This  limited  the  usefulness  of  the  performance  data. 
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SINCGARS  V  FOTE 


Background 

The  SINCGARS  V  is  a  single  channel,  jam  resistant  radio  that  has  the 
capability  for  membership  in  up  to  six  communications  nets.  Jam  resistance  is 
achieved  mainly  through  microprocessor-controlled  frequency  hopping.  It  has 
the  capability  for  transmitting  both  voice  communications  and  digital  data. 
Entry  of  frequency  and  other  types  of  data  is  accomplished  by  the  use  of  an 
alpha-numeric  keypad  and  other  controls. 

SINCGARS  V  is  being  developed  as  a  replacement  for  the  current  family  of  US 
Army  radios.  There  are  two  basic  versions  of  the  system,  one  for  deployment  in 
rotary  wing  aircraft  and  a  ground  version  that  can  be  either  backpacked  or 
installed  in  a  vehicle.  The  upcoming  operational  test  is  a  FOTE  of  the  ground 
version. 

One  of  the  key  issues  for  this  FOTE  is  training  effectiveness.  ARI  personnel 
at  Ft.  Hood  revised  the  original  TRADOC  training  issue  statement  to  more 
adequately  address  two  key  questions:  how  well  the  troops  were  able  to  perform 
their  tasks  before,  at  the  mid-point,  and  after  the  FOTE,  and  how  could  the 
training  time  be  reduced.  Essex  personnel  were  requested  to  assist  in  the 
development  of  a  training  effectiveness  evaluation  plan  to  be  used  by  ARI 
during  the  FOTE  that  would  collect  data  to  answer  these  questions. 

Objectives 


The  objective  of  this  effort  was  to  develop  a  realistic,  task-based  perfor¬ 
mance  test  for  assessing  the  effectiveness  of  the  SINCGARS  V  operator  training. 
The  test  was  to  measure  how  well  troops  performed  the  essential  tasks  before,  at 
the  mid-point,  and  at  the  end  of  the  FOTE,  and  to  determine  whether  the  training 
time  could  be  reduced  without  effecting  the  quality  of  the  training. 

Method 


Four  basic  steps  were  utilized  to  produce  the  test  methodology. 

First,  Essex  personnel  held  meetings  with  representatives  from  ARI  Fort  Hood 
and  the  TRADOC  Signal  school  to  define  the  specific  requirements  for  the  effort, 
essential  operator  tasks,  and  the  expected  training  content. 

Second,  Essex  personnel  reviewed  training  manuals,  training  materials,  POIs, 
and  the  literature  on  training  effectiveness  evaluation.  From  this  evolved  a 
list  of  specific  essential  tasks  on  which  the  operator  would  be  trained  and  a 
list  of  potential  methods  for  assessing  the  effectiveness  of  training.  The 
methods  were  reviewed  and  one  selected  that  was  most  appropriate  for  the 
SINCGARS  V  training  assessment. 

Third,  a  draft  methodology  was  developed  that  included  task  checklists. 

This  was  submitted  to  ARI  for  review  and  comment. 

Fourth,  the  draft  methodology  was  revised  based  on  comments  from  ARI  and 
submitted  as  a  final  product. 
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Product  description 

The  methodology  developed  for  the  training  effectiveness  evaluation  for  the 
SIh'CGARS  V  FOTE  consisted  of  the  following  four  step  process: 

One,  Administration  of  a  post-training  test.  The  trainees  would  perform 
each  step  of  each  essential  task  required  to  operate  the  equipment.  Performance 
times  would  be  measured,  along  with  an  assessment  of  whether  the  task  was 
performed  correctly  (e.g.,  step  performed,  step  performed  in  correct  sequence, 
etc . ) . 

Two,  Subjective  rating  of  training  effectiveness.  Subjective  rating  of 
training  effectiveness.  The  trainers  and  trainees  would  rate  the  training 
given  for  each  essential  task  as  effective  or  ineffective.  Comments  would  be 
elicited  on  deficiencies. 

Three,  Subjective  rating  of  sufficiency  of  training  time.  Trainers  and 
trainees  would  rate  the  sufficiency  of  the  amount  of  time  devoted  to  describing 
and  practicing  the  essential  tasks  as  not  enough,  about  right,  or  too  much. 

Four,  Comparison  of  ratings.  The  subjective  responses  obtained  above  would 
be  compared  with  observed  task  performances  in  the  posttraining  test. 

This  would  allow  a  determination  of  those  tasks  which  were  not  adequately 
explained  or  practiced.  To  assess  those  tasks  where  too  much  time  was  devoted 
would  require  direct  observation  in  the  instructional  setting. 

The  steps  listed  above  would  be  performed  three  times;  at  the  end  of 
training,  at  the  mid-point  of  the  FOTE,  and  at  the  end  of  the  FOTE.  The 
essential  tasks  that  were  identified  are  listed  in  Table  2. 

Table  2 

SINCCARS  V  Essential  Tasks 


1.  Prepare  Radio  for  Use 

2.  Operate  SINCCARS  V  in  the  single  channel  (SC)  mode 

3.  Prepare  the  SINCGARS  V  radio  for  frequency-hopping  (FH)  mode  operations 

4.  Communicate  with  SINCCARS  V  radio  net  in  the  frequency-hopping  mode 

5.  Join  a  SINCGARS  V  net  which  is  already  operating 

6.  Transmit  data  using  SINCGARS  V  radio  equipment 

7.  Operate  SINCGARS  V  radio  with  COMSEC  (KY-57) 

3.  Scan  RT  preset  channels-SC  mode 

9.  Use  SINCCARS  V  radio  relay  station  operations  (retransmission) 

10.  Retransmit  FH  control  instruction  data 

11.  Perform  SINCGARS  V  operator  maintenance  tasks 


O 


MSE  FOTE 


Background 


The  Mobile  Subscriber  Equipment  (MSE)  Non-Developmental  Item  (NDI)  acquisi¬ 
tion  is  one  of  the  more  critical  of  the  current  Army  programs.  The  intent  of 
the  program  is  to  purchase  from  industry  a  system  that  is  basically  "off-the- 
shelf"  and  can  replace  the  current  inventory  of  Army  tactical  telephone  systems. 
The  MSE  is  an  assemblage  of  S-250  and  S-250  extended  shelter-contained  com¬ 
munications  equipment  and  supporting  power  systems  that  can  provide  voice  and 
data  communications  support  for  a  notional  five  division  corps  deployed  in  an 
air-land  battle  scenario.  The  system  consists  of  up  to  42  node  center  switches 
(NCS)  connected  by  line-of-sight  (LOS).  Additional  equipment  includes  the  large 
extension  node  (LEN)  switches,  small  extension  node  (SEN)  switches,  radio  access 
units  (RAU),  mobile  subscriber  radio  terminals  (MSRT),  and  division  and  corps 
level  system  control  centers  (SCC).  A  more  detailed  system  description, 
provided  by  the  contractor,  is  included  in  Appendix  A  of  ARI  WP  FH  8801: 
Preliminary  Efforts  Directed  Towards  a  MANPRINT  Evaluation  of  the  Mobile 
Subscriber  Equipment  (MSE)~  4  Feb  88. 

The  MSE  is  intended  to  provide  the  user  with  a  secure,  automatic,  mobile, 
and  survivable  system  that  will  pass  data,  facsimile,  and  voice  traffic.  The 
MSE  will  interface  directly  with  Echelons  above  Corps  (EAC) ,  Combat  Net  Radio 
(CNR),  other  service,  NATO,  and  commercial  communications  systems.  The  system 
is  currently  being  developed  by  GTE  under  a  contract  that  calls  for  an  Initial 
Fielding/Follow-On  Operational  Test  and  Evaluation  (FOTE)  to  start  in  May  1988. 

Objectives 


The  initial  objective  of  this  support  was  to  provide  the  MANPRINT  input  for 
the  Independent  Evaluation  Plan  (IEP).  Later,  the  support  objectives  were 
expanded  to  include  tasks  for  the  preparation  of  a  MANPRINT  comprehensive 
continuous  evaluation  (C2E)  plan  for  the  MSE,  and  the  collection  and  analysis 
of  data  from  three  pre-FOTE  events  that  represented  opportunities  for  performing 
C2E.  These  three  events  were  the  following: 

•  The  System  Control  Center  (SCC)  man-machine  demonstration  and  hands-on 
exercise , 

•  The  cosite  interference  test  (CIT), 

•  The  field  installation  verification  (FIV)  demonstration. 

Method 


The  methods  used  for  performing  each  of  the  subtasks  in  this  task  are 
briefly  described  below. 

MANPRINT  IEP  Development.  The  IEP  Is  one  of  the  more  critical  of  the 
documents  associated  with  an  operational  test.  It  defines  the  issues,  criteria, 
evaluation  methodology,  and  data  required  to  perform  the  evaluations  and  answer 
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Che  Issues.  The  IEP  becomes  the  template  for  all  the  other  test  documentation, 
such  as  the  Test  Design  Plan  (TDP)  and  the  Detailed  Test  Plan  (DTP). 

The  MANPRINT  portions  of  the  MSE  IEP  were  developed  in  four  steps.  The 
first  step  involved  a  review  of  the  current  test  documentation.  This  included 
USAOTEA  Memorandum  10-3,  the  TRADOC  issues  and  criteria,  various  contractor 
supplied  system  descriptions,  and  the  existing  portions  of  the  MSE  IEP. 

The  second  step  involved  a  refinement  of  the  issues  and  criteria.  The 
existing  TRADOC  criteria  were  clarified  by  the  inclusion  of  a  more  detailed, 
precise  description  of  the  intent  of  the  criteria  in  terms  of  human  performance. 
They  were  also  expanded  to  include  USAOTEA  supplied  criteria,  specifically  in 
the  domains  of  personnel  and  human  performance  measurement. 

The  third  step  was  to  define  the  techniques  required  to  evaluate  the  issues 
and  criteria,  including  the  data  required  for  evaluation,  and  develop  a  draft 
MANPRINT  section  for  the  IEP.  This  was  submitted  to  ARI  and  USAOTEA  for  review 
and  comment. 

The  final  step  was  iterative  and  involved  a  succession  of  revisions  of  the 
MANPRINT  section  based  on  ARI  and  USAOTEA  comment  and  on  insights  gained  from 
Essex  personnel  during  C2E  activities.  A  final  revision  was  submitted  to  ARI 
and  USAOTEA  in  September  1987  for  incorporation  into  the  MSE  IEP. 

Development  of  the  C2E  Plan.  As  in  any  research  effort,  the  first  step  in 
the  development  of  the  MSE  MANPRINT  C2E  plan  is  to  review  the  relevant  documen¬ 
tation.  This  consisted  of  reviewing  the  IEP,  various  contractor  test  plans  and 
schedules,  the  MSE  System  MANPRINT  Management  Plan  (SMMP),  USAOTEA  Memorandum 
10-3,  and  attending  MSE  conferences.  It  became  evident  during  this  effort  that 
there  was  insufficient  guidance  available  for  the  content  and  format  of  a  C2E 
plan.  The  available  documentation  was  reviewed  to  determine  what  events  were 
upcoming  in  the  production  lead  time  activities,  what  data  could  possibly  be 
obtained  during  each  event,  and  what  the  current  MANPRINT  concerns  were  for  the 
MSE. 

From  this  review,  a  draft  plan  was  developed  that  specified  events,  expected 
data,  and  a  rough  schedule  for  data  collection.  This  plan  was  very  general  and 
designed  to  elicit  more  concrete  input  from  ARI  and  USAOTEA.  Based  on  their 
comments,  a  second,  more  detailed,  draft  plan  was  developed  and  submitted  for 
ARI  and  OTEA  approval. 

C2E  Data  Collection.  Three  data  collection  efforts  were  undertaken  for  the 
C2E  of  the  MSE.  These  involved  evaluating  demonstrations  of  different  aspects 
of  the  system  design  by  the  contractor  and  the  program  manager's  office  (PMO). 
The  demonstrations  were  the  System  Control  Center  (SCC)  Man-Machine  Demonstra¬ 
tion  and  Hands-On  Exercise,  the  Mobile  Subscriber  Radiotelephone  Terminal  (MSRT) 
Cosite  Interference  Test  (CIT),  and  the  MSRT  Field  Installation  Verification 
(FIV). 
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The  SCC  Man-Machine  Demonstration  and  Hands-On  Exercise  took  place  at  the 
contractor  site  in  Massachusetts,  where  the  PMO  was  conducting  a  demonstration 
of  the  system  using  three  representative  operators.  The  SCC  demonstration 
equipment  consisted  of  SCC  Command  shelter  equipment: 

(2  each  -  Visual  Display  Units  (VDUs)  with  keyboards, 

(1  each  -  Graphics  Display  Unit  (GDU)  with  operator's  console, 

(1  each  -  UGC-74B  teletype  with  keyboard),  Technical  Shelter  equipment 
(2  each  -  UGC-74B  teletypes, 

(the  duty  teletype  with  keyboard, 

(the  logbook  teletype  without  keyboard, 

(all  required  processor  bay  equipment  and  peripherals  needed  for  the 
demonstration, 

(NCS  simulator  that  was  used  to  return  acknowledgement  and  status  reports 
during  the  scenario. 

Data  collection  consisted  of  recording  participant  demographics,  taking 
notes  and  collecting  handout  information  during  classroom  instruction,  making 
observations  during  the  hands-on  exercise,  and  recording  interview  comments 
during  the  debriefing  interview.  Data  analysis  consisted  of  a  review,  sum¬ 
marization,  and  categorization  of  all  collected  information,  and  the  identifica¬ 
tion  of  potential  problems  associated  with  the  anticipated  SCC  design. 

The  MSRT  C1T  was  conducted  at  Ft.  Huachuca,  Arizona,  by  the  PMO  to  determine 
whether  the  MSRT  and  typical  CNRs  interfered  with  each  other  when  colocated. 

The  basic  design  of  the  CIT  consisted  of  the  use  of  two  jeeps  equipped  with  a 
MSRT  and  a  CNR,  and  two  jeeps  equipped  only  with  a  CNR.  The  jeeps  acted  as 
teams,  each  team  was  composed  of  an  MSRT  equipped  jeep  and  a  CNR  only  equipped 
jeep.  One  team  was  stationary  and  the  other  was  mobile,  placing  calls  at 
certain  distances  from  the  stationary  team.  The  distance  between  the  MSRT 
antenna  and  the  colocated  CNR  antenna  was  varied  as  one  of  the  independent 
variables . 

The  objectives  of  this  C2E  effort  were  to  glean  any  data  that  might  provide 
answers  to  existing  MANPRINT  issues  and  concerns,  and  obtain  insights  into 
potential  human  performance  problems  that  need  to  be  examined  during  the  FOTE. 
The  MANPRINT  data  collection  was  performed  using  two  methods;  observation  by  an 
experienced  MANPRINT  person,  and  a  structured  interview  with  the  operators  at 
the  conclusion  of  the  test.  The  observations  were  performed  primarily  during 
the  pretest  training  and  at  intervals  during  the  test,  and  were  used  to  develop 
items  for  the  structured  interviews.  Four  operators  were  drawn  from  the 
expected  target  audience,  but  the  structured  interviews  could  be  conducted  with 
only  two  of  them. 

The  MSRT  FIV  was  conducted  at  Fort  Hood,  in  conjunction  with  an  antenna 
pattern  test.  The  objective  of  Essex  attendance  at  the  FIV  was  to  look  for 
potential  MANPRINT  domain  problems  to  be  studied  during  the  upcoming  FOTE. 
Contractor  personnel  viewed  the  FIV  solely  as  a  means  of  verifying  the  adequacy 
of  the  Installation  instructions  in  their  draft  technical  bulletins.  The  31 
Victor  MOS  personnel  (three  in  a  team)  performed  most  of  the  actual  work,  but 
were  closely  observed  and  guided  by  contractor  personnel.  Once  the  installation 
was  completed,  contractor  personnel  checked  the  system  to  ensure  that  all 
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connections  were  properly  made,  and  then  removed  the  equipment.  The  antennas 
were  left  in  place  for  the  antenna  pattern  test  which  was  conducted  at  another 
site.  The  Mobile  Subscriber  Radio  Terminal  (MSRT)  was  not  employed  in  the 
antenna  pattern  test.  Rather,  a  separate  signal  generator,  not  a  part  of  the 
system  Itself,  was  used  for  this  test. 

The  MSRT  was  not  placed  in  operation  during  the  FIV.  Only  one  MSRT  was 
available,  so  it  would  not  have  been  possible  to  determine  whether  the  system 
was  functioning.  One  demonstration  of  initialization  of  the  system  with  the 
stand-alone  kit  was  conducted.  The  primary  purpose  of  the  demonstration  was 
simply  to  verify  the  instructions  to  the  operator.  Because  a  number  of  persons 
were  observing  the  demonstration,  it  was  not  possible  to  ask  questions. 

Results 


The  following  paragraphs  briefly  describe  the  results  of  each  of  the  MSii 
subtasks . 

MANPRINT  IEP  Development.  The  MSE  IEP  described  the  issues  and  criteria  for 
testing  the  system,  and  how  the  data  would  be  analyzed  to  answer  the  issues. 

The  MANPRINT  portion  of  the  MSE  IEP  contained  four  subissues  and  a  number  of 
attendant  criteria.  These  are  listed  in  Table  3,  along  with  USAOTEA  comments. 
For  each  of  these  subissues,  the  analysis  methods  emphasized  the  need  for 
objective,  human  performance  data  as  much  as  possible.  Writing  the  IEP  also 
provided  the  following  insights  into  the  requirements  for  developing  the 
MANPRINT  parts  of  an  IEP. 

•  First,  it  became  evident  that  some  domains  of  MANPRINT  are  very 
difficult  to  test.  This  is  particularly  true  for  manpower.  There 
needs  to  be  research  into  methods  for  collection,  analysis,  and 
evaluation  of  data  relative  to  the  impact  of  a  system  on  the  manpower 
requirements  of  units  and  the  Army  as  a  whole.  Iu  many  cases  it  may  be 
necessary  for  this  type  of  data  collection  to  be  performed  after 
fielding,  as  part  of  a  C2E  effort.  A  field  unit  probably  is  not  the 
appropriate  place  to  respond  to  the  total  manpower  issue. 

•  Second,  a  very  important  aspect  of  the  preparation  of  the  MANPRINT 
parts  of  an  IEP  is  integrating  MANPRINT  with  the  remainder  of  the  IEP 
sections.  The  MANPRINT  data  requirements  should  be  embedded  in  all  the 
other  sections  as  necessary,  especially  the  performance  issues. 

MANPRINT,  as  an  issue,  cuts  through  all  others  and  should  not  be 
treated  as  a  totally  separate  element.  The  data  for  MANPRINT  will 
usually  be  collected  during  the  scenarios  and  trials  done  for  the  other 
issues,  not  in  a  stand  alone  manner.  To  ensure  that  MANPRINT  is  treated 
adequately,  the  human  performance  data  collection  must  be  embedded 
within  all  the  relevant  issues. 

•  Third,  while  there  is  a  tremendous  and  justifiable  interest  in  more 
objective  human  performance  measurement  in  operational  test,  it  is  very 
difficult  to  collect  this  type  of  data  currently.  This  is  due  to  two 
factors.  One,  there  are  few  techniques  or  methods  currently  available 
that  are  appropriate  for  Army  operational  tests.  There  needs  to  be  more 
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research  to  develop  methodology  for  human  performance  measurement  in 
operational  testing.  Two,  for  the  current  and  future  methods  to  work, 
the  test  design  must  be  better  controlled  i  d  -'re  oriented  towards  the 
measurement  of  human  performance.  Currently,  the  emphasis  tends  toward 
designing  the  test  for  RAM  and  hardware  performance,  with  inadequate 
thought  given  towards  how  to  maximize  human  performance  data  collection. 

Development  of  the  C2E  Plan.  The  C2E  plan,  as  delivered,  provided  a  general 
management  document  for  USAOTEA  to  pursue  the  MANPRINT  C2E  of  the  MSE.  The 
acquisition  cycle  was  divided  into  two  phases,  the  Production  Lead  Time  (PLT) 
and  the  Follow-On  Test  and  Evaluation  (FOTE).  Table  4  illustrates  potential 
data  sources  that  were  identified  in  the  Plan,  by  phase.  Figure  1  illustrates 
the  MSE  C2E  process.  Table  5  lists  the  initial  MANPRINT  issues.  These  issues 
were  drawn  from  the  System  MANPRINT  Management  Plan  (SMMP)  and  the  IEP.  Table  6 
provides  a  matrix  of  data  collection  events  by  MANPRINT  issue.  It  became 
apparent  during  the  development  of  this  plan,  that  it  had  not  been  started  early 
enough  in  the  development  cycle  (e.g.,  PLT)  of  the  system.  Many  potential  data 
collection  events,  including  the  three  discussed  on  page  23,  were  occurring 
before  the  plan  was  completed.  Obviously,  a  C2E  plan  must  be  developed  at  the 
very  beginning  of  the  acquisition  cycle  and  revised  as  more  is  learned. 
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Table  3 


MANPRINT  Issues  and  Criteria  From  the  MSE  IEP 


Support  Issue  4.  Manpower  and  Personnel  Integration  (MANPRINT)  Elements. 

Subissue  1:  Does  the  training  provide  necessary  knowledge  and  skills  for 
users,  operators,  and  maintenance  personnel  to  employ  and  maintain  the  MSE 
System? 

USAOTEA  Comment:  The  major  concern  here  is  whether  the  pre-test  training  is 

sufficient  to  prepare  the  operators  and  maintainers  for  performing  their 
required  tasks  in  an  operational  field  environment.  Implicit  in  this  issue  is 
the  determination  of  whether  human  performance  problems  either  measured  or 
observed  during  the  field  test  can  be  attributed  to  Inadequate  training  or 
design  deficiencies  in  the  MSE  hardware  and  software. 

Criteria: 

Upon  completion  of  the  pretest  training  in  the  individual  and  collective 
training  plan,  80  percent  of  the  users,  operators,  and  maintainers  will  be  able 
to  perform  90  percent  of  the  critical  tasks  to  the  prescribed  standard  outlined 
in  the  training  material. 

At  least  80  percent  of  the  crews  must  complete  100Z  of  the  crew  critical  tasks. 

USAOTEA  Comment:  In  general,  "critical  tasks"  are  understood  to  be  those  which 
are  essential  to  successful  mission  performance  and  are  of  high  difficulty 
level  (hence  high  error  probability).  Crew  critical  tasks  are  understood  to  be 
those  performed  collectively  by  personnel.  These  critical  tasks  have  yet  to  be 
identified. 

The  training  program  and  support  material  must  provide  sufficient  information 
to  plan,  install,  maintain,  operate,  and  supervise  the  MSE  System  in  the 
judgement  of  80  percent  of  the  key  players  and  key  test  personnel. 


14 


Table  3  (cont.) 

MANPRINT  Issues  and  Criteria  From  the  MSE  IEP 


Subissue  2:  Are  the  MSE  signal  battalions  properly  organized  to  provide 
communications  support  for  the  command  and  control  of  support  units? 

USAOTEA  Comment:  The  major  concern  here  is  whether  there  are  adequate  numbers 

of  qualified  personnel  to  allow  for  proper  MSE  Signal  Battalion  manning  and 
organization.  The  issue  could  be  restated  as  follows:  Are  the  MSE  Signal 
Battalions  manned  by  sufficient  numbers  of  troops  having  the  prerequisite 
aptitudes  and  skill  levels?  This  issue  represents  the  MANPRINT  domains  of 
manpower  and  personnel. 

Criteria: 

No  additional  personnel  are  required  for  any  MSE  assemblage  to  meet  mission 
requirements  in  the  judgement  of  at  least  80  percent  of  the  key  players  and  key 
test  personnel. 

No  additional  personnel  are  required  to  command,  control,  plan,  install, 
operate,  maintain,  or  manage  the  MSE  network  in  the  judgement  of  80  percent  of 
the  key  players  and  key  test  personnel. 

No  additional  NRI  equipment  or  personnel  are  required  to  provide  interface  from 
Combat  Net  Radio  system  (SINCCARS  and  AN/VRC-12  series  radio)  in  the  judgement 
of  80  percent  of  the  key  players  and  key  test  personnel. 

USAOTEA  COMMENT:  NRI  equipment  is  not  normally  a  part  of  the  MANPRINT  assess¬ 
ment,  but  will  be  conducted  by  the  MANPRINT  personnel  during  the  MSE  FOTE. 

Operation  and  maintenance  of  the  MSE  assemblages  does  not  require  skills  and 
aptitudes  greater  than  those  typical  of  personnel  representing  the  full  range 
of  characteristics  of  the  designated  MOS.  The  Military  Occupational  Skill 
(MOS)  categories  designated  should  permit  the  selection  and  classification  of 
MSE  operators  and  maintainers  having  the  necessary  aptitudes,  skills,  and 
qualities  to  1 )  perform  the  required  tasks,  and  ?)  be  trained  to  an  acceptable 
level  of  performance  on  these  tasks  under  all  likely  operational  conditions. 
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Table  3  (cone.) 


MANPRINT  Issues  and  Criteria  From  the  MSE  IEP 


Subissue  3:  Do  the  human  factor  engineering  (HFE)  aspects  of  the  MSE  System 
assemblages  provide  for  reliable  and  efficient  operation  and  maintenance  by 
representative  soldiers? 

Criteria: 

Operators  and  maintainers  will  be  provided  with  adequate  workspace  to  perform 
their  mission. 

USAOTEA  Comment:  This  will  include  functional  layout  of  components  of  the 
system  within  the  shelter,  as  well  as  all  aspects  of  habitability  such  as 
lighting,  noise,  heating/air  conditioning/ventilation,  etc. 

Adequate  storage  space  for  TMs ,  tools,  parts,  personal  gear,  individual  weapons, 
and  all  other  items  necessary  to  perform  the  mission  is  provided. 

Centrals,  displays,  and  other  equipment  will  be  operable  by  representative 
soldiers  while  wearing  cold  weather  clothing  (Zone  6  with  arctic  gloves  in  a 
non-shelterized  environment)  and/or  Nuclear-Biological-Chemical  MOPP  IV 
protective  clothing  (for  use  in  shelterized  assemblages). 

Components  are  easily  accessible  to  maintenance  technicians. 

The  MSE  System  will  be  designed  to  enable  installation,  operation,  and  transport 
IAW  good  human  factors  engineering  principles. 

The  man-machine  interface  will  be  user  friendly  to  allow  the  performance  of  the 
mission. 

The  capabilities  of  the  MSE  assemblages  must  be  compatible  with  that  required 
by  the  users  to  perform  the  necessary  tasks  and  missions. 

Human  performance  of  the  expected  operational  and  maintenance  tasks  is  not 
degraded  by  any  aspect  of  the  design  of  the  human-system  interface. 
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Table  3  (cont.) 


KANPRTNT  Issues  and  Criteria  From  the  MSE  IEP 


Subissue  4:  Is  the  MSE  System's  equipment  safe  for  representative  soldiers 
to  operate  and  maintain? 

USAOTEA  Comment:  USAOTEA  understands  that  all  relevant  health  and  safety  issues 
are  to  be  considered.  These  will  include  such  problems  as  fire,  radiation, 
explosions,  cutting/blunt  surfaces,  environmental  extremes,  and  electrical 
shock. 

Criteria: 

The  MSE  System  equipment  and  assemblages  will  not  pose  any  unacceptable  safety 
or  health  hazard  to  the  operator  or  the  maintainer .  "Unacceptable"  is  defined 
as  a  critical  catastrophic  safety  or  health  hazard  as  defined  by  MIL-STD-882 . 

USAOTEA  Comment:  Critical  hazards  are  those  likely  to  cause  severe  illness  or 
injury  or  major  system  damage.  Catastrophic  hazards  are  those  likely  to  result 
in  death  or  system  loss. 

Hazards  that  are  an  unavoidable  part  of  the  system  configuration,  operating 
conditions,  etc.,  must  have  adequate  safeguards  to  minimize  the  likelihood  of 
injury  to  personnel. 
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Table  4 


Potential  Data  Sources  for  MSE  by  C2E  Phase 


PHASE  OF  SYSTEM  LIFE  CYCLE 

Data  Source _ PLT  Initial  Fielding 

Previous  Test  Results  X 

(OT,  Predecessor 
System's  Software) 

Shelter  Product  Assurance  X 

Test  (SPAT) 

Product  Assurance  Test  and  X 

Evaluation  (PATE) 

Product  Reliability  X 

Assurance  Test  (PRAT) 


Technical  Manuals  X  X 

Safety  Reports/Releases  X  X 

Other  Contractor  Documents  X 

AMSAA  IER  X 

Human  Factors  Engineering  X 

Analysis  (HFEA) 

Health  Hazards  Assessment  X 

(HHA) 

Destination  Final  Acceptance  X 

Testing  (DFAT) 

Doctrine  and  Tactics  Training  X 

New  Equipment  Training  (NET)  X 

Field  Training  Exercise  (FTX)  X 

Follow-on  Operational  Test  and  X 

Evaluation  (FOTE) 

USAOTEA  IER  X 
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Figure  1.  Functional  flow  of  Major  Activities  for  the  MSE  C2E  Process 


Table  5 


MSE  MANPRINT  C2E  Issues,  By  Domain 


1.  Manpower. 

a.  Assemblage  Personnel.  Are  additional  personnel  required  for  any  MSE 
assemblage  to  meet  mission  requirements? 

b.  Network  Personnel.  Are  additional  personnel  required  to  command, 
control,  plan,  install,  operate,  maintain,  or  manage  the  MSE  network? 

c.  IDS  Maintenance  Personnel.  Will  there  be  sufficient  numbers  of  IDS 
maintenance  personnel  for  repair  of  the  C-E  equipment  in  the  MSE  Signal 
Battalion? 

2.  Personnel. 

a.  Adequacy  of  Skills.  Does  operation  and  maintenance  of  the  MSE  as¬ 
semblages  require  skills  and  aptitudes  greater  than  those  typical  of  personnel 
representing  the  full  range  of  characteristics  of  the  designated  MOS? 

b.  Feeder  MOS  Impact.  What  will  be  the  impact  of  the  feeder  MOS'  not  being 
required  to  possess  the  objective  aptitudes  (Electronics,  General  Technical, 
Reading  Grade  Level)  for  transition  to  the  MSE  MOS? 

c.  Unit  Level  Maintenance.  Will  the  operators  be  capable  of  executing  unit 
level  maintenance  on  assigned  equipment? 

3.  Training 

a.  Transfer  of  Knowledge.  Does  the  training  provide  necessary  knowledge 
and  skills  for  users,  operators,  and  maintenance  personnel  to  employ  and 
maintain  the  MSE  System? 

b.  Individual  Performance.  Upon  completion  of  the  pretest  training  in  the 
individual  and  collective  training  plan,  can  80  percent  of  the  users,  operators, 
and  maintainers  perform  90  percent  of  the  critical  tasks  to  the  prescribed 
standard  outlined  in  the  training  material? 

c.  Crew  Performance.  Can  at  least  80  percent  of  the  crews  complete  100%  of 
the  crew  critical  tasks  after  collective  training? 

d.  Training  Material  Content.  Do  the  training  program  and  support  material 
provide  sufficient  information  to  plan,  install,  maintain,  operate,  and 
supervise  the  MSE  System? 
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Table  3  (coat.) 

MSE  MANPRINT  C2E  Issues,  By  Domain 


4.  Human  Factors  Engineering 

a.  Workspace.  Are  Operators  and  maintainers  provided  with  adequate 
workspace  to  perform  their  mission? 

b.  Storage  Space.  Are  adequate  storage  space  for  TMs ,  tools,  parts, 
personal  gear,  individual  weapons,  and  all  other  items  necessary  to  perform  the 
mission  provided? 

c.  MOPP  IV,  Cold  Weather  Operations.  Are  controls,  displays,  and  other 
equipment  operable  by  representative  soldiers  while  wearing  cold  weather 
clothing  (Zone  6  with  arctic  gloves  in  a  non-shel terized  environment)  and/or 
Nuclear-Biological-Chemical  MOPP  IV  protective  clothing  (for  use  in  shelterized 
assemblages) ? 

d.  Accessability .  Are  components  easily  accessible  to  maintenance  tech¬ 
nicians? 

f.  Compatibility.  Are  the  capabilities  of  the  MSE  assemblages  compatible 
with  that  required  by  the  users  to  perform  the  necessary  tasks  and  missions? 

g.  Design  for  Human  Performance.  Is  human  performance  of  the  expected 
operational  and  maintenance  casks  degraded  by  any  aspect  of  the  design  of  the 
human-system  interface? 

h.  Tongue  Weight.  Will  the  tongue  weight  of  the  PU-753  (equal  to  153  lbs) 
prohibit  use  of  the  5th  percentile  female  as  MSE  personnel? 

5.  System  Safety  and  Health  Hazards. 

a.  Safety  and  Hazards.  Do  the  MSE  System  equipment  and  assemblages  pose 
any  unacceptable  safety  or  health  hazard  to  the  operator  or  the  maintainer? 
"Unacceptable"  is  defined  as  a  critical  catastrophic  safety  or  health  hazard  as 
defined  by  MIL-STD-882. 

b.  Safeguards.  Do  hazards  that  are  an  unavoidable  part  of  the  system 
configuration,  operating  conditions,  etc.,  have  adequate  safeg"ards  to  minimize 
the  likelihood  of  injury  to  personnel? 
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Table  6 


Anticipated  MSE  C2E  Eata  Returns  by  Major  Event  and  MANPRINT  Issue 


Training 

3a  Transfer  of  Knowledge 
3b  Individual  Performance 
3c  Crtv  Performance 
3d  Training  Material  Content 
HFE 

4a  Workspace 

4b  Storage  Space 

4c  MOFP  IV,  Cold  Operations 

4d  Accessibility 

4e  Canpatibllity 

4f  Design  for  limai  Performance 

4g  Tongue  Weight 

Safety/m 
5a  Safety 

5b  Health  Hazards 


X  X  X  X  X 

XXX 
XXX 
X  X  X  X 

oxox  xxxx 

0  0  0  0  X  X 

0  X 

0  XXX 

XXX 
X  X 

X  X 

0  X  0  X  X 

0  X  0  X  X 


Nates:  *  Future  indicates  that  data  to  complete  the  assessnent  of  this  Issue  will 
rrm>  frcm  events  beyond  the  POIE. 

X  means  that  data  can  be  acquired,  0  means  that  only  indications  of  potential  Issues  can  be  acquired 
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C2E  Data  Collection 


For  each  of  the  HSE  C2E  events  monitored  by  Essex  personnel,  the  data  return 
was  not  as  great  as  expected.  This  was  primarily  due  to  the  MANPRINT  personnel 
not  being  in  the  role  of  controller  of  data  collection.  The  events  were 
organized  by  the  PMO  and  the  objectives  were  to  demonstrate  aspects  of  the 
system  design  other  than  MANPRINT.  In  all  cases,  concerns  regarding  the  design 
of  the  MSE  were  raised.  These  are  briefly  described  below. 

SCC  Han-machine  demonstration  &  hands-on  exercise.  Based  on  the  SCC  Man- 
Machine  Demonstration  and  Hands-On  Exercise,  it  appears  that  U.  S.  Army  Signal 
Corps  personnel  can  effectively  use  the  SCC  for  network  management  in  terms  of 
opening,  moving,  and  closing  network  elements  and  their  associated  links; 
however,  the  efficiency  and  accuracy  of  their  performance  may  suffer  from 
usability  and  readability  of  reports,  messages,  VDU  screens,  and  the  GDU 
graphics.  The  capability  of  the  operators  to  perform  these  functions  in  the 
shelter-mounted  equipment  and  in  a  battlefield  environment  cannot  be  assessed 
from  the  data  collected.  The  capability  of  the  operators  to  perform  functions 
of  the  SCC  other  than  the  three  mentioned  above  is  also  unknown. 

A  major  concern,  based  on  the  results  of  this  event,  is  the  overall  quality 
of  the  operator-machine  interface.  A  number  of  items  were  identified  that 
indicate  that  this  interface  is  less  than  ideal  at  this  point  in  its  design 
cycle.  The  primary  items  of  concern  that  were  identified  are  as  follows. 

•  Compatibility  of  labeling  of  the  equipment,  and  the  screen/report 
formats  with  American  stereotypes  and  U.  S.  Army  expectations  appear  to 
be  questionable.  The  current  displays  are  based  on  French  and  precursor 
system  design,  and  labels  have  been  translated  from  French. 

•  The  quality  of  the  GDU  display  appeared  marginal  in  producing  crisp, 
clear  images  with  adequate  brightness. 

•  Graphics  coding  on  the  GDU  is  very  complex  and  must  be  fully  reviewed 
by  HFE  and  Signal  Corps  subject  matter  experts. 

•  Hardcopy  reports  were  not  available  in  the  appropriate  shelters  where 
such  reports  function  as  an  operator  aid.  Useful  hard  copy  reports  are 
not  available  for  printout  at  all  the  shelters  that  could  use  them. 

•  Definition  of  functional  tasks  and  their  allocation  to  various  crew 
stations,  especially  in  terms  of  crew  workload  must  be  verified  as 
reflecting  the  U.  S.  Army  doctrine  and  operating  philosophies  for  the 
network . 

Cosite  Interference  test.  The  cosite  interference  test  produced  a  low  data 
return  for  MANPRINT.  In  general,  the  physical  size  of  the  terminal  was  larger 
than  expected,  taking  up  a  large  part  of  the  back  of  the  jeep.  The  control  and 
display  surface  was  cleanly  designed,  with  little  visual  clutter.  The  test  data 
from  the  CIT  seemed  to  support  the  original  hypothesis  of  an  interference 
problem  between  the  MSRT  and  the  CNRs . 


Based  on  the  results  of  this  MANPRINT  data  collection  effort  on  the  MSRT, 
there  are  a  number  of  concerns  that  need  to  be  further  examined  during  the 
FOTE.  Each  is  discussed  below. 

•  First,  it  is  obvious  that  the  cosite  interference  problem  has  direct 
relevance  to  training  and  doctrine.  The  doctrine  regarding  the 
placement  of  CNRs  relative  to  the  MSRTs  will  need  to  be  established  and 
introduced  into  the  training  for  the  operators  and  users  of  the  MSRTs. 
This  issue  needs  to  be  followed  up  with  data  collection  during  all 
phases  of  the  test,  especially  phase  III  when  the  unit  is  free  playing. 

•  Second,  it  was  observed  that  the  operator  had  to  input  user  identifica¬ 
tion  and  frequency  data  through  the  DSVT/DNVT  handset  without  visual 
feedback  of  correct  data  entry.  Without  the  benefit  of  a  display  for 
visual  reference,  many  errors  could  be  introduced  by  the  operators.  If 
the  MSE  system  does  not  have  a  suitable  error  checking  capability  built- 
in,  data  on  the  frequency  of  errors  and  the  impact  of  these  errors  on 
unit  communication  needs  to  collected  during  all  phases  of  the  FOTE. 

•  Third,  during  the  training  it  was  indicated  that  occasionally  there 
would  be  a  loss  of  dial  tone.  When  this  occurred,  the  operator  was  to 
tap  the  switchhook  on  the  DSVT/DNVT  or  turn  the  system  off.  This  loss 
of  dial  tone  needs  to  be  further  explored  from  both  a  RAM  standpoint 
and  to  determine  if  this  leads  to  loss  of  incoming  calls. 

•  Fourth,  it  was  observed  that  the  indicator  lights  washed  out  in  bright 
sunlight  and  the  audible  alarm  frequency  is  the  same  as  the  incoming 
call  signal.  While  the  operators  reported  little  difficulty  with  this, 
given  the  limited  nature  of  the  exercise  the  indicator  light  brightness 
and  the  audible  alarm  frequency  need  to  be  explored  more  to  determine  if 
there  could  be  any  behavior  impact  on  the  operators. 

Field  installation  verification.  The  FIV  was  an  exploratory  effort  by  the 
PMO  to  look  at  the  installation  concepts  for  the  MSRT  in  various  vehicles,  the 
hardware  required  for  installation,  the  installation  procedures,  and  the 
manuals.  The  following  potential  MANPRINT  issues  were  observed  during  the  FIV 
and  need  to  be  explored  more  fully  during  the  FOTE. 

•  The  manuals  were  difficult  to  use  due  to  steps  being  out  of  sequence, 
drawings  not  being  in  scale,  lack  of  exploded  drawings,  and  part 
numbers  referenced  in  the  manuals  not  matching  the  actual  part  numbers. 

•  There  were  a  number  of  problems  with  the  vehicle  installations.  In  the 
M151  Jeep,  the  antenna  mount  was  on  the  road  side  and  in  front  of  the 
driver.  The  antenna  could  not  be  adequately  tied  down  without  danger 
of  it  shorting  out  when  it  couched  metal.  In  the  M1008  CUCV,  the  DSVT 
was  strapped  to  the  front  seat  between  the  driver  and  the  passenger. 
This  precluded  a  third  person  riding  in  the  cab.  In  the  M113  Personnel 
Carrier,  the  MSRT  mounting  extended  forward  of  the  passenger  back  seat 
rest.  This  could  lead  to  injury  to  personnel. 
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Noise  levels  in  tactical  vehicles  tend  to  be  high.  These  levels  could 
mask  the  sound  on  the  incoming  call  alert. 


•  Vehicle  mounting  holes  locations  were  determined  by  using  a  tape 
measure.  Drilling  templates  would  be  much  more  accurate. 

•  There  was  no  installation  tool  kit.  Several  times  the  installers  were 
observed  to  borrow  tools  from  personal  tool  k its,  such  as  a  right  angle 
drill.  A  complete  installation  tool  kit  should  be  supplied  with  the 
equipment . 

•  Spares  of  parts  such  as  bolts  and  nuts  were  not  included  in  the 
installation  kits.  Installers  were  observed  obtaining  parts  from 
hardware  stores  to  replace  lost  or  damages  parts.  The  installation 
kits  should  include  spares. 
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M1A1  Main  Battle  Tank  FOE 


Background 


The  Ml  Main  Battle  Tank  has  been  evolving  over  the  last  ten  years.  The 
latest  design  is  designated  the  M1A1  Abrams  and  resembles  its  predecessor  in 
most  respects.  The  design  changes  consist  primarily  of  the  following: 

•  Substitution  of  a  120mm  main  gun  for  the  105mm  main  gun.  The  120mm  gun 
uses  combustible-case  ammunition,  and  requires  a  new  weapon  mount  and 
redesigned  fire  control  system. 

•  Improved  armor  protection. 

•  Improved  suspension  system,  transmission  and  final  drive. 

Forty-one  (41)  M1A1  Abrams  were  being  fielded  for  the  first  time  to  the  3rd 
Squadron,  3rd  Armored  Cavalry  Regiment  (ACR)  at  Ft.  Bliss,  Texas.  USAOTEA 
performed  a  FOE  of  the  M1AI  from  January  to  June  1987,  supported  by  ARI  in  the 
area  of  MANPRINT. 

The  FOE  used  an  expanded  sample  data  collection  (SDC)  methodology  to  evaluate 
M1A1  production  models  during  new  equipment  training  (NET)  and  unit  ARTEPS.  The 
primary  test  issues  were  as  follows. 

•  Can  the  M1A1  tank  main  gun  be  calibrated  using  procedures  prescribed  in 
the  "M1A1  Calibration  Policy,"  FC-17-12-1A1 ,  Tank  Combat  Tables? 

•  Have  the  materiel  deficiencies  disclosed  during  the  MlEl  OT  II  been 
corrected? 

•  Can  the  M1A1  tanks  be  supported  with  planned  logistics  concepts? 

•  Can  the  MlAl  tank  crews  effectively  use  the  on-board  nuclear,  biologi¬ 
cal,  and  chemical  (NBC)  system? 

Objectives 


The  objectives  of  the  MANPRINT  assessment  were  to  provide  supporting  data 
and  information  to  address  the  test  issues  described  above. 

Method 


The  data  collection  methods  used  in  the  MlAl  FOE  consisted  of  subjective 
techniques.  It  was  originally  intended  for  there  to  be  some  human  performance 
data  collection,  but  the  noninterference  requirement  imposed  on  the  FOE  by  the 
host  unit  would  not  allow  any  instrumentation  of  side  tests.  Therefore,  data 
collection  relied  on  structured  interviews,  questionnaires,  task  checklist 
completion,  and  debriefs. 


26 


The  data  were  collected  In  three  stages.  During  January  and  February,  1987, 
demographic  data  and  training  observations  were  collected  during  the  NET. 
Post-ARTEP  debriefs  of  the  crews  were  conducted  during  April  1987,  along  with 
limited  observations.  At  the  end  of  the  FOE,  during  June  1987,  interviews  were 
conducted  with  the  test  directorate,  unit  commanders,  and  data  collectors.  In 
addition,  final  questionnaires  were  administered  to  the  crews. 

Results 


The  MANPRINT  efforts  during  the  M1A1  FOE  focused  on  the  second  test  issue: 
were  the  MANPRINT  deficiencies  found  during  previous  testing  of  the  M1EL  during 
OT  II  corrected?  Of  the  29  deficiencies  previously  reported,  11  (38%)  were  not 
corrected  in  the  MlAl.  This  included  the  following: 

«  No  provision  for  human  waste  elimination  and  disposal  during  prolonged 
buttoned-up  operations. 

•  Inadequate  storage  space  for  personal  gear. 

•  The  NBC  system  does  not  protect  the  crew  from  Carbon  Monoxide  (CO)  gas, 
it  only  removes  the  odors  (propellent  fumes?)  associated  with  the  gas. 
This  makes  it  all  the  more  insidious  a  threat. 

•  The  noise  from  the  NBC  system  interferes  with  conversation  over  the 
intercom  system. 

•  Driver's  seat  provides  inadequate  back  support. 

According  to  the  questionnaire  and  interview  data,  the  crews  could  calibrate 
the  main  gun  rapidly  and  effectively.  Supporting  data  from  gunner  accuracy  was 
not  available.  The  only  major  negative  comment  regarding  the  main  gun  system 
indicated  that  there  was  considerable  drift,  requiring  correction  every  15 
minutes  or  so.  The  crew  indicated  that  the  drift  was  much  worse  than  the  M60's 
105mm  gun. 

The  major  reported  difficulty  with  the  logistics  concept  was  in  the  resupply 
capabilities.  The  crews  indicated  that  the  platoons  needed  about  50%  more 
personnel  and  transportation  resources  to  effectively  resupply  the  MlAl  platoons 
during  extended  missions. 

In  “Edition  to  the  lack  of  protection  from  CO  mentioned  above,  a  number  of 
other  problems  were  identified  relating  to  the  on-board  NBC  system.  These 
included  the  following: 

•  The  Units'  Standard  Operating  Procedure  (SOP)  needs  to  be  revised  to 
accommodate  the  new  NBC  system. 

•  Training  for  the  NBC  system  needs  to  be  expanded  to  include  how  the 
system  works  and  how  to  troubleshoot. 
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•  The  NBC  system  exhaust  on  the  left  side  of  the  tank  is  very  hot,  and 
can  cause  burns  to  personnel  or  possible  fires.  This  needs  to  be 
protected  or  vented  better. 

Additional  findings  from  the  MANPRINT  evaluation  of  the  M1A1  during  the  FOE 
include  the  following: 

•  The  heater  tends  to  fill  with  dust.  The  impact  of  this  is  unknown  at 
present . 

•  The  front  fuel  caps  do  not  lock  open  while  fueling.  They  become  an 
inconvenience  and  can  fall  on  a  crew  member's  wrist  and  cause  injury. 

•  New  equipment  training  (NET)  was  rated  good.  It  was  felt  that  a 
specialized  NET  should  be  added  for  those  personnel  who  were  already 
19K  MOS.  They  were  trained  on  topics  that  were  consistent  with  their 
previous  Ml  experience.  They  felt  that  this  was  redundant  and  a  NET 
for  only  those  differences  was  a  better  approach. 

•  It  was  felt  that  the  scout  personnel  should  have  some  MlAl  training  to 
make  them  familiar  with  the  capabilities  of  the  system,  because  they 
have  to  work  closely  with  the  MlAl  units. 
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DISCUSSION 


Essex  Corporation,  through  ARI  Fort  Hood,  provided  MANPRINT  test  and 
evaluation  support  to  USAOTEA  for  four  operational  systems:  AN/TRC-I70, 

SINCGARS  V,  MSE,  and  the  MlAl.  This  support  included  definition  of  data 
collection  methodology,  data  collection,  data  reduction  and  analysis,  and 
report  preparation.  In  addition,  Essex  personnel  worked  towards  defining 
methods  for  the  development  of  MANPRINT  and  C2E  plans,  human  performance 
measurement  in  operational  testing,  and  field  training  assessment.  The  validity 
of  some  of  these  methods  will  be  determined  in  future  testing. 

There  were  a  number  of  issues  regarding  MANPRINT  operational  test  and 
evaluation  that  introduced  difficulties  to  the  test  teams  and  should  be  briefly 
discussed  here. 

The  first  issue  involved  is  measuring  human  performance  during  Army  opera¬ 
tional  testing,  one  of  the  goals  of  USAOTEA.  Based  on  the  experiences  of 
MANPRINT  personnel  during  this  contract,  there  are  a  number  of  difficulties  in 
meeting  this  goal.  First,  to  reliably  collect  human  performance  data,  the  test 
design  must  lend  itself  to  human  performance  measurement  under  experimentally 
controlled  conditions.  As  experienced  during  the  MlAl  FOE  and  the  AN/TRC-170 
FOTE,  MANPRINT  considerations  were  frequently  identified/addressed  too  late  or 
not  considered.  In  many  cases,  the  tests  were  designed  exclusively  to 
accommodate  RAM  or  hardware  testing. 

Second,  manual  human  performance  measurement  requires  tremendous  personnel 
resources.  As  illustrated  during  the  AN/tk.C-170  FOTE,  these  resources  were 
limited  and  impacted  the  quality  of  the  data  collected.  The  obvious  solution 
is  to  use  automated  data  collection  methods,  but  this  remains  difficult  for 
U.S.  Army  operational  test  and  evaluation  due  to  the  variety  of  systems  tested, 
the  environment  in  which  they  are  tested,  and  the  costs  of  test  equipment  and 
instrumentation . 

Third,  in  order  for  MANPRINT  data  to  be  of  high  quality,  MANPRINT  personnel 
reed  to  be  involved  with  the  test  planning  from  the  very  beginning.  As  was 
illustrated  by  the  MSE  C2E  plan  development  subtask,  all  too  often  test  events 
can  eclipse  the  planning.  MANPRINT  personnel  need  to  define  their  test 
requirements  in  terms  of  issues  and  criteria,  experimental  design,  instrumenta¬ 
tion,  and  resources  very  early  in  the  test  cycle  in  order  for  them  to  be 
adequately  integrated  into  the  other  test  requirements.  Prospects  for  success 
can  be  increased  by  taking  an  aggressive  lead  in  test  planning,  but  receptive 
test  agency  personnel  are  also  essential. 

The  final  issue  involves  methodology.  With  the  advent  of  the  MANPRINT 
initiative,  a  number  of  test  requirements  became  highly  visible  in  operational 
testing.  These  included  manpower,  personnel,  training,  and  human  performance. 
It  became  apparent  during  the  development  of  the  MANPRINT  portion  of  the  MSE 
IEP  that  cu.rent  methods  were  not  adequate  for  collecting  quality  data  on  these 
requirements.  This  was  further  reinforced  by  the  experiences  on  the  AN/TRC-170 
and  SINCGARS  V  tests.  To  adequately  answer  the  test  issues  associated  with  the 
domains  of  MANPRINT,  new  techniques  and  methods  for  data  collection,  reduction, 
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analysis,  and  evaluation  need  to  be  developed.  The  ideal  solution  would  be  a 
set  of  procedures  for  MANPRINT  operational  test  and  evaluation  that  provide 
guidance  on  how  to  perform  data  collection  and  analysis  adaptable  to  a  wide 
variety  of  system  configurations  and  applications.  These  procedures  would 
vastly  improve  the  quality,  reliability,  validity,  and  comparability  of  MANPRINT 
data  from  operational  test  and  evaluation. 
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